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The common practice of using of guided notes in the post-secondary classroom is not fully 
appreciated or understood. In an effort to add to the existing research about this phenomenon, the 
current investigation expands on previously published research and one previously published meta¬ 
analysis that examined the impact of guided notes on post-secondary student achievement. 
Specifically, this study examines the different variables that moderate the effect of using guided 
notes in the classroom, the impact of guided notes relative to professor-provided notes or student¬ 
generated notes, and unlike previous studies, the present meta-analysis, includes both published and 
unpublished research and some previously unexamined variables. Results indicate that overall, 
guided notes can produce a moderate impact on student achievement. The study discusses the 
implications and limitations of this research. 


Student Engagement 

The concept of student engagement has received 
considerable attention in the last two decades in 
response to declines in academic achievement and 
student motivation (Fredricks, Blumenfeld & Paris, 
2004). Engaged students can be described as students 
who are actively participating in their college learning 
experience (Bomia et al., 1997). Astin’s (1984) theory 
of student involvement suggests that students who are 
actively engaged in the learning process experience 
greater learning, more personal growth, and increased 
satisfaction, and they are more likely to be retained. 
Unfortunately, many post-secondary classes today 
encourage passive learning, as opposed to active 
learning, through traditional lecture presentations. 
According to Leong (2006), these traditional lecture- 
based classes are no longer the “most effective” (p. 66) 
approach in reaching today’s student. Guided notes 
have been suggested as one low-cost and “low-tech” 
(Heward, 1994) approach for bridging the gap between 
the professor’s lecture and the students’ level of 
engagement. 

Heward (1994) maintains that guided notes can 
facilitate increased student focus and engagement in 
lectures. Guided notes, according to Heward (1994), are 
defined as “teacher-prepared handouts that ‘guide’ a 
student through a lecture with standard cues and prepared 
space in which to write key facts, concepts, and/or 
relationships” (p. 304). These note guides are provided to 
students as handouts, electronically, or as PowerPoint 
slides, in which main ideas are included as prompts with 
blank space for students to fill in additional information. 
Guided notes have been shown to provide a number of 
beneficial effects in the post-secondary learning arena, 
including increased student verbal engagement in lectures 
(Austin, Lee, Thibeault, Carr, & Bailey, 2002; Rieland, 
2008) and a higher quality of note-taking (Austin, Lee, & 
Carr, 2004; Neef, McCord, & Ferreri, 2006). 


Post-secondary students, according to Austin et al. 
(2004), are “notoriously poor note takers” (p. 314) and 
were found to record only about 50% of the main ideas 
being presented during lectures. In response to this, 
some instructors have tried giving students complete 
lecture notes, but complete notes do not encourage 
students to become more actively engaged (Cook, 2009; 
Konrad, Joseph, & Eveleigh, 2009). However, these 
researchers found that when guided notes were 
introduced, students became more engaged in the 
lectures by becoming more active responders to the 
lecture and the prompts provided by the guided notes. 
Subsequently, with the provision of guided notes, 
Austin, et al. found that the quality of the note taking 
improved. Neef et al. (2006) found that when provided 
with guided notes, students recorded 90% of the lecture 
information correctly. Research has demonstrated that 
the quality of note-taking has been found to be related 
to increased student achievement (Hamilton, Seibert, 
Gardner, & Talbert-Johnson, 2000; Neef et al., 2006; 
Peverly et al., 2007; Williams & Eggert, 2002). 

A number of researchers have examined the impact 
of guided notes for post-secondary students as an 
approach to improve student engagement, and 
subsequently, student achievement (Austin et al., 2004; 
Austin et al., 2002; Bahadourian, Tam, Greer, & 
Rousseau, 2006; Katayama, 1997; Katayama, Crooks, 
& Weiler, 2000; Ketchum, 2007; Lawson, Bodle, & 
McDonough, 2007; Lewis, 2009; McCann, 2008; 
Musti-Roa, Kroeger, & Schumacher-Dyke, 2008; 
Narjaikeaew, Emarat, & Cowie, 2009). And, although 
guided notes have been found to have a positive effect 
in most studies, the impact of guided notes is not 
always found to be significant or positive. This makes it 
very difficult to draw clear-cut conclusions about the 
impact of guided notes in post-secondary education. A 
meta-analysis of these studies would be an appropriate 
and effective approach to synthesizing and integrating 
the sometimes conflicting results from this quantitative 
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research, and it would provide a general measure of the 
impact of guided notes on student achievement. It also 
would be beneficial to an overall understanding of the 
impact of guided notes on student achievement. The 
present project will conduct such a meta-analysis. 

Glass, McGaw, and Smith (1981) define meta¬ 
analysis as the “analysis of analyses.” Glass et al. 
(1981) propose the use of meta-analysis as a means for 
effectively aggregating any number of studies in the 
research literature. With this technique, the findings of 
a number of smaller research studies can be pulled 
together to enhance the overall sample size and 
statistical power. It is a method by which studies that 
might not otherwise be easily integrated can be 
compared and contrasted with each other. 

A review of the existing literature reveals that only 
one study has attempted to synthesize the existing 
research examining the impact of guided notes on post¬ 
secondary student achievement (Konrad et ah, 2009). 
This investigation of post-secondary data was limited to 
three published studies; however, these three studies 
indicate a relatively positive impact. 

Publication bias is a concern when performing a 
meta-analysis (Wolf, 1986). Publication bias occurs 
when studies that find significant results for an effect 
being investigated are more likely to be published than 
studies that do not find significant findings. Publication 
bias has the potential of inflating the effect size 
estimates (Glass et ah, 1981; Hedges, 1986; Hunter & 
Schmidt, 2004; Rosenthal, 1979), and therefore it is 
important that unpublished information be included 
when conducting a meta-analysis. 

Thus, the current meta-analysis addresses the issue 
of publication bias by including all available studies on 
the impact of guided notes on post-secondary student 
achievement. The current investigation includes 12 
studies, seven (58.3%) of which were published and 
five (41.6%) of which were not published. These 
research studies include published, peer-reviewed 
journal articles and theses. In doing so, this current 
study is able to do a thorough investigation of this 
phenomenon by examining all available research for 
inclusion in the synthesis; therefore, more appropriately 
representing the population of quantitative research on 
the impact of guided notes on post-secondary student 
achievement. 

Method 

There are a number of recommended procedures 
for conducting a meta-analysis (e.g., Glass et al., 1981; 
Hedges, 1986; Hunter & Schmidt, 2004; Rosenthal, 
1979; Wolf, 1986) which offer slight variations from 
one another but essentially share much in common. 
Glass et al. (1981) recommends the following steps for 
conducting a meta-analysis. First, studies should be 


gathered on the topic or phenomena of research. The 
studies which can be included in the meta-analysis must 
fit within the defined parameters for analysis, while 
representing as much of the population of data available 
on the research area. The research must be quantitative. 
Glass et al. (1981) maintain that a thorough search must 
be conducted of the subject area. This step can 
potentially introduce the “most serious form of bias” (p. 
57) into the meta-analysis, because it is difficult to 
evaluate the impact of the search bias. The more 
exhaustive the search, the more likely it is that search 
bias will be minimized. 

The next step, according to Glass et al. (1981), is to 
describe, classify, and code all the research studies to 
be included in the meta-analysis. In this step, 
measurement consistency is imperative. Glass et al. 
(1981) suggest that studies should be coded 
independently, so that inter-rater agreement can be 
established. The moderator variables that have been 
included for consideration must be clearly defined so 
that raters are able to make clear distinctions between 
the various classifications. For the purposes of this 
meta-analysis, a random sample of studies was coded at 
least twice in order to establish the reliability of the 
coding procedures. Moderator variables have been 
tested for inter-rater reliability and were found to be 
reliable classifications k = .95 of the time. 

The final step in performing the meta-analysis, 
according to Glass et al. (1981), is the analysis of the 
overall mean effect size measures and the mean effect 
size measures for each moderator being examined. 
Once the effect size measures have been calculated, 
interpretation and reporting of results follows. 

Sample of Studies 

Studies included in this meta-analysis were 
obtained initially through an extensive electronic 
search. Various electronic databases were searched over 
a six month period of time. These include Academic 
Search Complete, Digital Dissertations, Educational 
Resources Information Circuit (ERIC), EBSCO, 
Electronic Journal Center, JSTOR, GoogleScholar, and 
Psychlnfo. The search examined research spanning the 
years 1980-2011. The descriptive search criteria 
employed to identify related materials included such 
combinations as guided notes, partial notes, and note 
guides, as well as each of these criteria with the 
addition of college, university, or college students. 
Similar to the Konrad et al.’s (2009) meta-analysis, an 
article did not have to use the term “guided notes” 
specifically; however, the article was included only 
when the notes being used adhered to the definition by 
Heyward (1994). Abstracts of articles were inspected, 
and those articles that did not appear to meet the initial 
inclusion criteria were discarded. The inclusion criteria 
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were: (a) articles examining the use of guided notes 
with post-secondary students, (b) research that includes 
a measure of academic performance, and (c) research 
focused on the use of guided notes during lecture in 
comparison to no guided notes, full lecture notes, or 
some other note strategy, and finally, (d) research 
making use of some form of experimental or quasi- 
experimental design. 

The literature meeting the criteria that was 
electronically available was printed, and other sources 
were ordered through the Youngstown State University 
and the Kent State University library systems. Next, the 
reference list of each article was searched in an effort to 
find any additional pertinent studies. All obtained articles, 
dissertations, presentations, and project reports were 
reviewed, and those primary-level studies which included 
the participant population and treatment population of 
interest, as well as the necessary statistical information, 
were included in this meta-analysis. In all, more than 35 
studies were identified by these methods, and examined 
for possible inclusion in this meta-analysis. 

There were a few studies obtained through this 
search process that initially appeared as suitable 
candidates for inclusion, but careful inspection revealed 
that they did not meet the criteria discussed above. Many 
of the articles simply did not provide any usable data and 
presented results qualitatively. Additionally, a number of 
studies suggested that effects were measured; however, 
only average test scores were provided (without 
necessary standard deviations or group sample sizes). In 
order to include studies in a meta-analysis, data must be 
sufficiently reported so that an effect size can be 
calculated. Specifically, studies must provide sufficient 
descriptive and inferential data, such as means, standard 
deviations, sample sizes, variances, t tests,/tests, as well 
as chi-square information, in order to allow for the 
calculation of effect sizes. If the necessary descriptive 
and inferential data were not provided, an attempt was 
made to contact the author of the study in an effort to 
acquire such information. Studies which failed to provide 
the necessary information, either in the original form or 
via supplemental data provided by the author, were 
excluded from the meta-analysis. Once studies were 
examined and studies were eliminated for insufficient 
data, a total of 12 useable studies remained for inclusion 
in the current investigation. From these 12 studies, 27 
independent effect sizes were calculated. These studies 
comprise a total of 1,529 participants. 

Coding of Studies 

Each study was coded according to the following 
information: (a) year of study, (b) source of research 
study, (c) type of intervention, (d) the research design, 
(e) the type of course, (I) sample size of the study, and 
(g) the achievement measurement provided. Student 


level (undergraduate or graduate) was considered, but 
all of the useable studies included undergraduate 
students. The primary data and study characteristics are 
maintained in The Statistical Package for the Social 
Sciences (SPSS 18). 

Year of Study 

The first study characteristic is the year of the 
research publication. Studies included in this meta¬ 
analysis include research from 1997 through 2011. 
Publication year was organized into half-decade 
categories of: 1997-2000, 2001-2005, and 2006-2011. 

Source of Research 

The second study characteristic is the source of the 
research. Studies included in this meta-analysis can be 
classified as published journal articles, unpublished 
dissertations/thesis, and unpublished presentations. As 
indicated above, this meta-analysis differs from prior 
meta-analytic studies on guided notes (e.g., Konrad et 
ah, 2009) in that it includes research on guided notes 
from published and un-published sources. 

Intervention Type 

Research studies were categorized as having one of 
two different interventions. Some studies compared 
data from groups using their own notes, relative to 
groups using guided notes, while other studies looked at 
student groups who used complete lecture notes, 
relative to groups using guided notes. Complete lecture 
notes, for the purpose of this investigation, are copies of 
the professor’s lecture notes that are provided to the 
students prior to lecture. 

Research Design 

Research design categories include random 
assignment to multiple groups with a control group and 
without a control group, and non-random assignment to 
multiple groups with a control group and without a 
control group, or single group designs with pre and post 
test measures. 

Type of Course 

The fifth study characteristic of this meta-analysis 
is type of course. Studies were categorized according to 
the discipline area described as providing the course. 

Sample Size 

The sixth characteristic of this meta-analysis is 
sample size. This describes the reported sample size for 
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each study. Sample sizes were organized into the 
following categories: (1) less than 30 participants, (2) 
30 to 100 participants, and (3) more than 100 
participants. 

Achievement Measure 

A last study characteristic for this meta-analysis is 
achievement measure used. Achievement measure 
categories include (a) course grade point average, or 
final grade; (b) final test or quiz score; (c) an exam; 
and/or (d) a quiz score. Achievement measures were 
categorized as one of these four types according to the 
study author(s’) descriptions. The dependent variable 
for all studies is the measure of student achievement 
provided by the authors. For most studies, authors have 
provided a mean achievement measure score for the 
students from the experimental group and for the 
students in the control groups. 

Research Questions 

The present meta-analysis examines the following 
questions: 

1. What is the impact of the use of guided notes 
on student achievement in post-secondary 
courses when compared to students who do 
not receive guided notes? 

2. Does the impact of guided notes differ by the 
publication year of the study? 

3. Does the level of impact of guided notes differ 
by the source of the study (e.g., dissertation or 
theses, journal article, presentation, internet 
posting, project report)? 

4. Does the impact of guided notes differ 
depending on the intervention type (e.g., 
student notes vs. guided notes, or full lecture 
notes vs. guided notes)? 


5. Does the impact of guided notes vary by 
research design (e.g., random assignment to 
multiple groups with a control group)? 

6. Does the impact of guided notes differ by the 
type of course (i.e., discipline area)? 

7. Does the impact of guided notes differ by the 
sample size of the study? 

8. Does the impact of guided notes differ by type 
of achievement measure used (e.g., final 
course grade, final exam, unit quiz, pre-post 
test change)? 

Calculation of Effect Sizes 

There are a number of methods for calculating 
effect sizes (Glass et al., 1981; Hedges & Olkin, 1985; 
Hunter & Schmidt, 2004; Rosenthal, 1979; Wolf, 
1986). For this meta-analytic study, all statistics from 
each study will be converted to Hedges d. Hedges d 
statistic is defined as the difference between the means 
of the experimental and control groups divided by the 
inter-group standard deviation. The use of the inter¬ 
group, or pooled standard deviations, is used in an 
effort to reduce positive bias that can result with 
computing effect size measures (Grissom & Kim, 
2005). The overall effect size measures and research 
category effect size measures were calculated by hand 
and with comprehensive meta-analysis (Biostat, 2009; 
see http://www.meta-analysis.com/), a software 
package used for meta-analytic review of research. 

Effect size measures were calculated with means 
and standard deviations using the formula from Johnson 
(1989). With this approach, the mean for the control 
group is subtracted from the mean for the experimental 
group, resulting in a mean difference, and this value is 
divided by the pooled standard deviation for both 
groups. The result is the effect size measure associated 
with that study, as demonstrated in the graphical 
representation found in Figure 1. 


Figure 1 

Graphical Representation of Effect Size Measure 



Control 


Treatment 


The space between the two distributions represents the "effect size" 
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Once the effect sizes are calculated for the 
individual studies, the overall effect size measure for all 
the studies combined can be calculated. This can be 
done, according to Glass et al. (1981), by simply 
calculating the mean of the individual effect size 
measures. However, this approach does not take into 
consideration the fact that the studies vary in sample 
size. Hedges and Olkin (1985) provide a formula for 
calculating the overall mean effect size as an unbiased 
weighted estimate (weighted by sample size) of the 
population effect size. With this approach, an overall 
effect is calculated by weighting individual studies by 
sample size, or the amount of information a study has to 
contribute, by using the inverse variance which is 
roughly proportional to the sample size. The overall 
mean effect sizes for this meta-analysis were calculated 
according to the procedures recommended by Hedges 
and Olkin (1985). 

Next, a homogeneity test was conducted in order to 
determine if all studies share a common effect size 
(Hedges & Olkin, 1985). The statistical test for the 
homogeneity of effect size tests the null hypothesis that 
states all effect sizes in the study are equal. If the test of 
homogeneity of the effect size measure, more 
commonly referred to as the Q statistic (Hedges & 
Olkin, 1985) reveals that the studies do not share a 
common effect size, the contrast between the means 
within and between each study characteristic is 
investigated. It is through this process that the impact of 
individual characteristics on the effect size measures 
can be more fully understood. 

Interpretation of Effect Sizes 

There are a number of different approaches to 
interpreting effect size measures (Glass et al., 1981; 
Hedges, 1986; Wolf, 1986). For the purpose of this meta¬ 
analysis, studies with several independent effect sizes 
were calculated as several individual samples (Glass et 
al., 1981; Kulik, 1983a, 1983b). This approach allows 
the researcher to integrate all of the available effect sizes 
in the meta-analysis, thus including as much information 
as possible in the final analysis. Effect size measures 
were calculated for each study. An overall mean effect 
size measure was calculated for the group of studies in 
the meta-analysis, and mean effect size measures were 
computed for each research category. This was 
accomplished using random effects models as opposed to 
fixed effects. The assessment data used in the current 
investigation was identified as a record of exam and quiz 
scores from varied situations; fixed effects models are 
used when all studies are assumed “functionally 
identical” (Borenstein, 2007). 

Post hoc analyses were conducted in order to 
examine specifically where significant differences exist 
between the mean effect size measures for each level in 


each research category. Post hoc analyses were also 
conducted to determine if significant differences exist 
within each of the levels of the categories. 

Results 

The purpose of this meta-analysis was to 
investigate the impact of guided notes on student 
achievement in postsecondary coursework. The study 
also examined a number of variables that could 
potentially impact or mediate this relationship. This 
meta-analysis included a number of studies identified 
by a computerized literature search across many 
disciplines. With a total of 12 useable studies, we 
calculated 27 independent effect sizes. These studies 
comprise a total of 1,529 participants. The range of the 
effect sizes is 1.67, with a minimum effect-size measure 
of -.008 and a maximum effect-size measure of +1.67. 
The overall mean effect measure for this group of effect 
sizes was d = 0.546, p < .001, a moderate effect size 
according to the rough standards established by Cohen 
(1977). 

These findings indicate that the use of guided notes 
can have a moderate impact on student achievement in 
post-secondary coursework. A 95% confidence level 
ranges from 0.342 to 0.749. This confidence interval 
does not contain the value of zero, implying that the 
treatment of using guided notes had a significant impact 
(Johnson, 1989). This effect size suggests that the 
average student participating in the guided notes 
conditions exceeds the academic achievement of 
approximately 72% of the students in the non-guided 
note conditions. Figure 2 presents a graphical 
representation of this impact of guided notes on student 
achievement. 

Twenty-one of the 27 effect sizes (77.7%) included 
in these analyses were positive, indicating that guided 
notes had a positive impact on student learning. These 
analyses also reveal that 14 (51.8%) of the 27 effect 
size measures had an effect size of 0.5 or greater, 
indicating that the effect of guided notes on student 
achievement was at least moderate. One study 
demonstrated neither a positive or negative effect. The 
remaining five studies (18.5%) revealed a negative 
effect, indicating that traditional approaches of having 
students generate their own notes or providing students 
with professor-generated full lecture notes produced a 
greater impact on learning than the guided notes. Table 
1 provides a breakdown of the studies meeting the 
inclusion criteria. 

The grand mean analyses also revealed a <9(26) 
=158.99, p < 0.001 statistic, indicating significant 
heterogeneity across the 27 effect size measures 
included in this investigation. Therefore, further 
analyses are necessary to understand the variegation in 
effect sizes across the different studies. Specifically, 
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Figure 2 

Graphical Representation of the Impact of Guided Notes on Student Achievement 
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Table 1 


Studies Included in Meta-Analysis 


Study 

n of ES 

ES range 

Austin et al. (2002) 

1 

0.734 

Austin et al. (2004) 

3 

0.987 to 0.989 

Bahadouria et al. (2006) 

2 

1.383 to 1.680 

Katayama et al (2000) 

3 

0.222 to 0.849 

Katayama (1997) 

4 

0.008 to 1.286 

Ketchum (2007) 

2 

1.286 to 1.409 

Lawson et al. (2007) 

2 

0.143 to 1.315 

Lewis (2009) 

1 

-0.142 

McCann (2008) 

2 

-0.342 to -.0370 

Musti-Roa et al. (2008) 

2 

.926 to 1.055 

Narjaikaew et al. (2009) 

1 

0.614 

Neef et al. (2006) 

4 

-0.248 to 0.312 


these analyses explore the potential relationship 
between study characteristics and effect-size measures 
to determine which study characteristics influence the 
effect-size measures and which do not. Further analyses 
were conducted to explore the individual research 
characteristics and their potential influence on effect- 
size measures in an effort to explain this inconsistency 
across the individual effect-size measures. 

This additional analysis revealed significant 
variation in the secondary variable year of research 
study, Q w = 0.668, p < 0.05. Specifically, a greater 
mean effect size is seen for the 2001-2005 period of 
research (d = 0.933) relative to the other study periods 
(1997-2000, d = 0.540; 2006-2011, d = 0.460). As 
demonstrated in Figure 3, examination of the year of 
research study characteristic reveals no substantive 
pattern in the data across the time periods investigated. 
Subsequently, the significant effect size difference may 
have resulted because of sample size differences across 
the 2001-2005 research period (« = 4) relative to the 
1997-2000 (n = 7) and 2006-2011 periods (n = 16). 


Another secondary moderator, type of control 
group, also revealed significant within group 
differences, Q w = 0.462, p < 0.05. Specifically, this 
moderator separated out whether the control group 
participants in each study generated their own notes {d 
= 0.781, n = 15) or had completed notes provided for 
them by their instructor (d = 0.265, n = 12). These 
results suggest that achievement results of students 
using guided notes were significantly higher when 
compared to students generating their own notes, 
relative to the achievement results of students using 
guided notes when compared to students who were 
provided with professor generated notes. Although both 
groups demonstrated a significant, positive effect (p < 
0.05) the impact of using guided notes had the greatest 
effect relative to the students generating their own 
notes. 

Additionally, significant variation was revealed for 
the primary variable, discipline area of the research, Q w 
= 0.432, p < 0.05. This moderator revealed that guided 
notes have a significant positive effect on student 
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Figure 3 

Scatter Plot of Year by Mean Effect of Guided Notes 


1.5 


I 


0.5 


♦ 

♦ 


1996 


-0.5 


1998 


2000 


2002 


2004 


2(^)6 


2008 ♦ 2010 

♦ 


achievement in the discipline areas of psychology (d = 
0.603), physics (d = 0.614), and education (d = 0.749); 
a significant negative effect on the achievement of 
students in mathematics (d = -0.355); and a non¬ 
significant negative effect for dentistry students (- 
0.142). 

Finally, the primary variable sample size of study 
revealed significant within group variation, Q w = 0.519, 
p < 0.05. Specifically, nine smaller sample studies ( n < 
30) revealed the largest significant impact (d = 0.892), 
followed by seven large sample studies (« > 101) 
revealing a large significant impacts {d = 0.631). 
Eleven studies with sample sizes of 31 to 100 
participants revealed a non-significant positive results 
(d = 0.231). A further breakdown of this moderator 
revealed that one study (Narjaikeaew et ah, 2009; d = 
0.614) had a sample size that was significantly larger 
than all the other studies (« = 1002). Since this was 
suspected to be an influential outlier, this study was 
removed, and the sample size moderator was re¬ 
analyzed. This process reduced the overall effect size 
measure {d = 0.498 relative to the original d = 0.546), 
but this reduction was not significant. The resultant 
analysis generated a minimal change in the mean effect 
size measure of the largest sample size group (n > 101) 
from the original effect of d = 0.631 to a effect of d = 
0.642. Figure 4 demonstrates a breakdown of the mean 
effect size measures across the different sample sizes of 
the studies (without Narjaikeaew et ah, 2009). 

Finally, no significant different effect-size 
measures were found across the categories for the 


primary variables of research design (e.g., 
experimental, repeated measure, quasi-experimental) 
and type of assessment (e.g., exam or quiz) or the 
secondary variable source of research (e.g., publication, 
presentation, or thesis) and published or not. Table 2 
presents the summary of these analyses. 

Discussion 

The purpose of the current investigation was to 
examine the impact of guided notes on post-secondary 
student achievement, by incorporating all available 
quantitative data through a meta-analytic synthesis of 
the existing research. To date, only one other meta¬ 
analysis has examined this research question, and as 
such, these researchers only incorporated three studies, 
all published studies, into their analysis (Konrad et ah, 
2009). Thus, the present study significantly expands the 
scope of what was examined, both in terms of research 
reviewed and moderator variables considered, over and 
above the one prior meta-analytic investigation of the 
impact of guided notes on student achievement in post- 
secondary-level education. 

One key addition of this meta-analytic 
investigation beyond the previous is the addition of 
non-published research. The Konrad et ah, (2009) meta¬ 
analysis incorporated only three research studies that 
appeared in peer-reviewed journals. Thorough 
inspection of the existing research revealed that there 
were not only more peer-review studies, but additional 
research studies that were not incorporated into the 
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Figure 4 

Mean Effect Size Measures by Sample Size of Study 


2 


1.5 


1 


0.5 


0 


-0.5 



Table 2 


Summary of Analysis Results Across Study Characteristics 


Variable and Categories 

Number of Effect 
Sizes (n) 

Within-Group 

Effects 

Mean Effect 
Size d 

Year of Study 

2006-2011 

16 

0.668* 

0.460* 

2001-2005 

4 


0.933* 

1997-2000 

7 


0.540* 

Study Source 

Peer-Review Publication 

15 

0.549 

0.644* 

Conference Presentation 

9 


0.320* 

Dissertation/Thesis 

3 


0.807* 

Type of Control Group 

Student Generated Notes 

15 

0.462* 

0.781* 

Full Lecture Notes 

12 


0.265* 

Study Research Design 

Experimental Design 

12 

0.596 

0.669* 

Repeated Measures Design 

14 


0.434* 

Quasi Experimental Design 

1 


0.734* 

Discipline Area 

Psychology 

12 

0.432* 

0.603* 

Physics 

1 


0.614* 

Mathematics 

2 


-0.355* 

Education 

11 


0.749* 

Dentistry 

1 


-0.142 

Type of Assessment 

Exam 

15 

0.547 

0.547* 

Quiz 

12 


0.547* 

Sample Size of Study 

0 to 30 

9 

0.519* 

0.892* 

31-100 

11 


0.231 

101+ 

7 


0.631* 
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prior meta-analysis, and five additional research studies 
that, while subjected to considerable review (refereed 
conference presentations and two Masters’ theses), 
were not published. As indicated above, it is 
problematic to include only published research (i.e., 
publication bias), as doing so will generally include 
only research that has shown to have significant impact, 
and can potentially inflate the resulting effect size 
measures (Glass et al., 1981; Hedges, 1986; Hunter & 
Schmidt, 2004; Rosenthal, 1979). The present study 
corrects for this potential bias by including both 
published and non-published studies. The analysis of 
the impact of guided notes on post-secondary student 
achievement was found not to be significantly different 
whether reported by published or non-published 
research, Q( 2) = 2.180, = 0.336, so this inclusion 

indicates that when all available research is included, 
publication bias does not present a significant threat. 

In addition to expanding on a previous meta- 
analytic study of the impact of guided notes on post¬ 
secondary student achievement, the current meta¬ 
analysis incorporates some additional dynamics. For 
example, the current meta-analysis examined the use of 
guided notes relative to student generated notes and full 
lecture notes. Four studies in the current study 
examined the impact of guided notes relative to full 
lecture notes (Katayama, 1997; Katayama et ah, 2000; 
Lewis, 2009; Neef et ah, 2006). Overall, guided notes 
were found to have a small positive effect on student 
grades relative to student who used full notes, d = .265, 
p < 0.001. This result was significantly smaller than the 
large impact of guided notes that was found, relative to 
student generated notes, on student achievement, d = 
0.781, p < 0.001. While the impact of guided notes 
relative to full lecture notes is small, the result is 
significant, suggesting that there is some benefit in 
requiring students to be more actively involved in the 
lectures which they attend. 

Another noteworthy contribution of the current 
meta-analysis is the incorporation of different 
disciplines. As indicated earlier, all research satisfying 
the stated inclusion criteria was incorporated into this 
investigation. Due to its limited scope, the research of 
Konrad et ah (2009) included only studies with 
psychology students. The current investigation includes 
research from psychology (Austin et ah, 2002; Austin et 
ah, 2004; Ketchum, 2007; Neef et ah, 2006), physics 
(Narjaikaew et ah, 2009), mathematics (McCann, 
2008), education (Bahadourian, 2006; Katayama, 1997; 
Katayama et ah, 2000; Musti-Roa, 2008), and dentistry 
(Lewis, 2009). The analysis revealed that significant 
differences existed across the disciplines, and careful 
examination demonstrated that a small negative impact 
was found in one study from the area of mathematics 
and dental hygiene. Both of these disciplines were 
represented by one study each. Potentially, these results 


reflect that guided notes are used less often in these 
disciplines and therefore guided notes are not used as 
effectively. On the other hand, the large positive effect 
size measure resulting for the other disciplines— 
psychology (d = 0.603, p < 0.001), physics (d = 0.614, 
p < 0.001), and education ( d = 0.749, p < 0.001)—were 
statistically equivalent. The large significant impact 
found in research from psychology students and 
Education students might reflect the fact that the two 
disciplines that are more closely associated with 
learning and cognition, and they are the two disciplines 
producing the preponderance of research in the arena of 
guided note use. The physics effect size measure was 
generated by one study in that discipline. 

Moreover, the current investigation incorporated a 
number of other variables not found in the previous 
meta-analysis. First, year of research is a commonly 
reported variable in meta-analytic research. For this 
investigation, research was found that ranged from 
1997 to 2011, a fourteen-year span that was divided 
accordingly: 1997-2000, 2001-2005, and 2006-2011. 
The analysis revealed that there was a significant 
difference in the effect sizes computed across these 
time periods; however, all three time periods revealed 
significant moderate-to-large effect sizes. Of interest is 
that the most current research, from 2008 (d = 0.288, p 
= 0.434) and 2009 (d = 0.256, p = 0.497), revealed the 
smallest and non-significant effect size measures. 

Another commonly included variable in meta- 
analytic studies is the sample size of the research study. 
The present analysis reveals significant differences 
were revealed, with the 0-30 and the 100 or more 
sample sizes resulting in large significant effect size 
measures ( d = 0.892, p < 0.001 and d = 0.631, p < 
0.001) respectively, while the 31 to 100 sample size 
group revealed a small non-significant effect size 
measure. Careful examination of the data discloses that 
one possible explanation for the difference is that the 
31-100 sample size group included 4 of the 5 negative 
effect size measures. 

Other moderator variables that were included in 
this study did not demonstrate significant differences 
across levels. Research design, as well as the form of 
assessment, did not reveal significant differences across 
the respective levels. Interestingly, the effect size 
measures for the two assessment levels (exams and 
quizzes) were statistically equivalent. 

Limitations 

There were several limitations associated with the 
present research study. A number of the individual 
constructs were represented in a small sample of 
studies. Although this might have occurred as the result 
of an insufficient computer literature search strategy, 
that is not the case for the current investigation. The 
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literature search process was thorough and exhaustive, 
and it revealed a number of additional empirical 
research studies not included in the past meta-analytic 
review of this subject area. For example, this initial 
limitation applies to the variable students’ education 
level. All of the available research on guided notes’ 
impact on post-secondary students’ achievement 
included research with undergraduate-level students 
while no graduate level studies were available. 
Additionally, there is considerable research available on 
students at the K-12 level of education; however this 
study focused on post-secondary education. 

Another limitation of the present study is that it is 
difficult to sufficiently break down moderator 
categories enough to examine as much information as 
possible without creating too much overlap in the 
results. For example, with the meta-analytic approach, 
the meta-analytic researcher is at the mercy of the 
authors who have conducted research in the area. The 
researcher has to rely on the authors or individual 
researchers to report results accurately, describe the 
studies well, report statistics appropriately, and respond 
to inquiries about their research if there are any 
questions or discrepancies (Larwin & Larwin, 2011). 
This results in another layer of concern that has to do 
with the selection of variables for possible examination 
in a meta-analysis. It would have been interesting to 
investigate variables such as gender or student 
characteristics (traditional versus non-traditional), and 
the examination of these kinds of variables may have 
revealed additional insight, complexities, and 
moderator effects about the effectiveness of guided 
notes on student performance. However, that data is not 
available, so these variables could not be examined. 

Finally, interpreting the results of meta-analytic 
studies such as this one can be challenging when the 
content area is not the focus of considerable 
quantitative research. There are a number of studies on 
the use of guided notes that examined their impact 
qualitatively (Badger, White, Sutherland, & Haggis, 
2001; Randolph, 2005; Toole, 2000) or which included 
very limited quantitative information (Carr, 2004; 
Kreiner, 1997; Lazarus, 1993). There were studies that 
found that students have very positive perceptions 
regarding guided notes (Yilmazel-Sahin, 2007) or 
found that guided notes are beneficial when used with 
low-performing students (Austin, 1999) or as a 
mechanism to encourage increased class participation 
(Rieland, 2008). While all of this research provides 
valuable insight as to the use of guided notes, these 
studies did not provide enough or appropriate data for 
inclusions. 

In spite of the challenges in conducting the 
current investigation, the results of this study, which is 
based on a current body of research, are noteworthy. 
The “low-tech” inclusion of guided notes into the 


post-secondary classroom can significantly impact 
student achievement. Many researchers have 
suggested that the increase in student achievement is 
due to an increase in student engagement (Austin et 
ah, & Bailey, 2002; Austin et ah, 2004; Neef et ah, 
2006; Rieland, 2008). More empirical research is 
needed in order to fully understand if and how the use 
of guided notes impacts student engagement. Also, 
while it would seem that engagement may be the 
mechanism by which guided notes do produce a 
positive impact on student achievement (Heward, 
1994), additional research is needed to further 
understand the relationship between guided notes and 
student achievement. 
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